TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras path, setup/packaging fixes #913

psamanoelton · 2025-11-07T00:43:08Z

Summary

Enable building TFQ against TensorFlow 2.16.2 with Bazel 6.5.x
Keep Keras 2 (“tf-keras”) path via TF_USE_LEGACY_KERAS=1 to avoid Keras 3 breakages
Allow selecting system/venv/conda Python; wire that interpreter through Bazel
Make wheel-build script honor chosen Python and include setuptools

Key changes

WORKSPACE: bump @org_tensorflow to 2.16.2, add a tiny @python local_repository shim
third_party/python_legacy/defs.bzl: export interpreter as a string (what TF’s local_config_python expects)
configure.sh: --python= flag; generate .tf_configure.bazelrc + .bazelrc + the python shim
.bazelrc: ensure TF_USE_LEGACY_KERAS=1 for build/test; keep existing warnings/rpath tweaks
release/build_pip_package.sh: invoke ${PYTHON_BIN_PATH:-python3} and ensure setuptools present
setup.py: (optionally) relax deps to run on TF 2.16 (sympy==1.14); extras pin tensorflow==2.16.0

Testing

Linux/Ubuntu 22.04 (Docker), Python 3.11, Bazelisk → Bazel 6.5.0
bazel build ... release:build_pip_package succeeds
Unit tests (scripts/test_all.sh) pass with
Wheel installs and imports with TF 2.16.2 + tf-keras 2.16.0, Cirq 1.3.0, NumPy 1.26.4

Notes

This keeps TFQ on Keras 2 path; Keras 3 migration can be handled in a a future PR.
This changes will help on the incoming upgrades to 2.20.

…h, setup/packaging fixes

mhucka

Thank you for the work, and I'm sorry about the number of changes requested here …

release/build_pip_package.sh

release/setup.py

configure.sh

WORKSPACE

mhucka · 2025-11-15T05:33:34Z

release/build_pip_package.sh

  fi

-  mkdir -p ${DEST}
+mkdir -p "${DEST}"


Trivial nit: something happened to the intendentation on this line.

Suggested change

mkdir -p "${DEST}"

mkdir -p "${DEST}"

mhucka · 2025-11-15T05:50:33Z

release/setup.py

+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",


We should omit these, since these versions are no longer actively supported.

Suggested change

"Programming Language :: Python :: 3.7",

"Programming Language :: Python :: 3.8",

"Programming Language :: Python :: 3.9",

mhucka · 2025-11-15T05:52:30Z

release/setup.py

+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
+        "Programming Language :: Python :: 3.10",
+        "Programming Language :: Python :: 3.11",


Let's also add 3.12 and 3.13 (assuming the updated TFQ from this PR works in those versions):

Suggested change

"Programming Language :: Python :: 3.11",

"Programming Language :: Python :: 3.11",

"Programming Language :: Python :: 3.12",

"Programming Language :: Python :: 3.13",

mhucka · 2025-11-15T06:02:17Z

release/setup.py

+    author="Google Inc.",
+    author_email="no-reply@google.com",


Since this file is getting changes for other reasons, it may as well also include an update to these values. The general approach we've taken in other major QAI projects is to use the following pattern, so I think it's worth including in this PR so that another PR isn't needed to make this change later.

Suggested change

author="Google Inc.",

author_email="no-reply@google.com",

author="The TensorFlow Quantum Authors",

author_email="tensorflow-quantum-team@google.com",

mhucka · 2025-11-15T06:06:42Z

tensorflow_quantum/__init__.py

 # pylint: enable=undefined-variable

-__version__ = '0.7.2'
+__version__ = '0.7.5'


Oops, sorry I didn't notice this before: the next version is actually 0.7.4 (since the current version on PyPI is 0.7.3):

Suggested change

__version__ = '0.7.5'

__version__ = '0.7.4'

mhucka · 2025-11-15T06:14:19Z

configure.sh


-function is_windows() {
-  # On windows, the shell script is actually running in msys
+is_windows() {


Suggested change

is_windows() {

function is_windows() {

mhucka · 2025-11-15T06:14:29Z

configure.sh


-function is_ppc64le() {
-  [[ "$(uname -m)" == "ppc64le" ]]
+write_legacy_python_repo() {


Suggested change

write_legacy_python_repo() {

function write_legacy_python_repo() {

mhucka · 2025-11-15T06:26:41Z

configure.sh

+  PY_ABS="$("${PY}" - <<'PY'
+import os,sys
+print(os.path.abspath(sys.executable))
+PY
+)"


Although this works, IMHO this construct is fragile because of the complicated syntax. (At first, I thought the )" by itself at the end was a typo.) Would the following not work?

Suggested change

PY_ABS="$("${PY}" - <<'PY'

import os,sys

print(os.path.abspath(sys.executable))

PY

)"

PY_ABS="$("${PY}" -c 'import os,sys; print(os.path.abspath(sys.executable))')"

mhucka · 2025-11-15T06:52:57Z

configure.sh

+"${PYTHON_BIN_PATH}" - <<'PY' || { echo "ERROR: tensorflow not importable by chosen Python."; exit 1; }
+import tensorflow as tf
+import tensorflow.sysconfig as sc
+print("TF:", tf.__version__)
+print("include:", sc.get_include())
+print("lib:", sc.get_lib())
+PY
+
+# --- discover TF include/lib robustly --------------------------------------
+HDR="$("$PYTHON_BIN_PATH" - <<'PY'
+import tensorflow.sysconfig as sc
+print(sc.get_include())
+PY
+)"
+
+LIBDIR="$("$PYTHON_BIN_PATH" - <<'PY'
+import os, tensorflow.sysconfig as sc
+p = sc.get_lib()
+print(p if os.path.isdir(p) else os.path.dirname(p))
+PY
+)"
+
+LIBNAME="$("$PYTHON_BIN_PATH" - <<'PY'
+import os, glob, tensorflow.sysconfig as sc
+p = sc.get_lib()
+d = p if os.path.isdir(p) else os.path.dirname(p)
+cands = glob.glob(os.path.join(d,'libtensorflow_framework.so*')) \
+     or glob.glob(os.path.join(d,'libtensorflow.so*')) \
+     or glob.glob(os.path.join(d,'_pywrap_tensorflow_internal.*'))
+print(os.path.basename(cands[0]) if cands else 'libtensorflow_framework.so.2')
+PY
+)"


The execution of this is slow because it imports tensorflow 4 times. It's true that the imports after the first time will be faster thanks to caching by the operating system, but still, it feels like there must be a more efficient way than running Python 4 times. Also, there is a little bit of duplication of code.

What about the following?

Suggested change

"${PYTHON_BIN_PATH}" - <<'PY' || { echo "ERROR: tensorflow not importable by chosen Python."; exit 1; }

import tensorflow as tf

import tensorflow.sysconfig as sc

print("TF:", tf.__version__)

print("include:", sc.get_include())

print("lib:", sc.get_lib())

PY

# --- discover TF include/lib robustly --------------------------------------

HDR="$("$PYTHON_BIN_PATH" - <<'PY'

import tensorflow.sysconfig as sc

print(sc.get_include())

PY

)"

LIBDIR="$("$PYTHON_BIN_PATH" - <<'PY'

import os, tensorflow.sysconfig as sc

p = sc.get_lib()

print(p if os.path.isdir(p) else os.path.dirname(p))

PY

)"

LIBNAME="$("$PYTHON_BIN_PATH" - <<'PY'

import os, glob, tensorflow.sysconfig as sc

p = sc.get_lib()

d = p if os.path.isdir(p) else os.path.dirname(p)

cands = glob.glob(os.path.join(d,'libtensorflow_framework.so*')) \

or glob.glob(os.path.join(d,'libtensorflow.so*')) \

or glob.glob(os.path.join(d,'_pywrap_tensorflow_internal.*'))

print(os.path.basename(cands[0]) if cands else 'libtensorflow_framework.so.2')

PY

)"

tf_output=$("${PYTHON_BIN_PATH}" - <<'PY'

import sys

import os

import glob

try:

import tensorflow as tf

import tensorflow.sysconfig as sc

except ImportError:

sys.exit(1)

print(sc.get_include())

lib_path = sc.get_lib()

lib_dir = lib_path if os.path.isdir(lib_path) else os.path.dirname(lib_path)

print(lib_dir)

cands = (glob.glob(os.path.join(lib_dir, 'libtensorflow_framework.so*')) or

glob.glob(os.path.join(lib_dir, 'libtensorflow.so*')) or

glob.glob(os.path.join(lib_dir, '_pywrap_tensorflow_internal.*')))

print(os.path.basename(cands[0]) if cands else 'libtensorflow_framework.so.2')

PY

)

if [[ $? -ne 0 ]]; then

echo "ERROR: tensorflow not importable by Python (${PYTHON_BIN_PATH})" >&2

exit 1

fi

{

read -r HDR

read -r LIBDIR

read -r LIBNAME

} <<< "${tf_output}"

mhucka · 2025-11-15T06:56:51Z

WORKSPACE


 load("@org_tensorflow//tensorflow:workspace0.bzl", "tf_workspace0")

-tf_workspace0()


Is the removal of this call to tf_workspace0() intentional?

psamanoelton added 3 commits November 6, 2025 14:44

TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras pat…

70c0578

…h, setup/packaging fixes

Commenting deprecated tf.patch

c208307

Updating build_pip_package.sh for properly handle python interpreter

a4283c7

mhucka self-assigned this Nov 11, 2025

mhucka added area/dependencies Involves libraries or other software that TFQ depends on area/health Involves general matters of project configuration, health, maintenance, and similar concerns area/installation Involves installation of TFQ area/tensorflow Involves TensorFlow labels Nov 11, 2025

mhucka requested changes Nov 13, 2025

View reviewed changes

psamanoelton added 9 commits November 13, 2025 17:13

Updating tutorial test tests and python linter

f8c0d5a

Applying linter to test tutorials

67d58d1

Solving Python Coding Style

48eb555

Solving Coding style and lint

28fba61

Fix yapf formatting

3f62fea

Applying yapf for formatting

38ae8ec

Solving comments

22b41b3

Fix PYthon lint

094b444

Adding nbclient for jntests

b5da73c

mhucka requested changes Nov 15, 2025

View reviewed changes

	"Programming Language :: Python :: 3.7",
	"Programming Language :: Python :: 3.8",
	"Programming Language :: Python :: 3.9",

	write_legacy_python_repo() {
	function write_legacy_python_repo() {


		load("@org_tensorflow//tensorflow:workspace0.bzl", "tf_workspace0")

		tf_workspace0()

TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras path, setup/packaging fixes #913

Are you sure you want to change the base?

TF2.16 support: hermetic python shim, TF 2.16.2 pin, legacy Keras path, setup/packaging fixes #913

Conversation

psamanoelton commented Nov 7, 2025

Uh oh!

mhucka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhucka Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mhucka Nov 15, 2025 •

edited

Loading